Multi-view scanning aerial cameras

ABSTRACT

An aerial camera for capturing images along two or more curved scan paths, the aerial camera comprising a scanning camera associated with each scan path, each scanning camera comprising an image sensor, a lens, a scanning mirror, and a drive coupled to the scanning mirror; wherein the drive is operative to rotate the scanning mirror about a spin axis according to a spin angle, the spin axis is tilted relative to the camera optical axis, the scanning mirror is tilted relative to both the camera optical axis and the spin axis and is positioned to reflect an imaging beam into the lens, the lens is positioned to focus the imaging beam onto the image sensor, and the image sensor is operative to capture each image along the scan path by sampling the imaging beam at a corresponding spin angle.

FIELD OF THE INVENTION

The present invention relates to high-performance multi-view aerialimaging systems and methods.

BACKGROUND OF THE INVENTION

Georeferenced aerial imagery, orthomosaics and 3D surface models areincreasingly used to visualize, analyze and manage the builtenvironment. Multiple views of each ground point from different angles,as well as high image resolution, are important both for visualizationand for high-fidelity 3D surface reconstruction. Since the builtenvironment undergoes constant change, aerial imagery and 3D surfacemodels are ideally updated on a regular basis. This motivates the use ofhigh-performance multi-view imaging systems that deliver high resolutionwhile minimizing operating cost.

Efficient imaging of large areas is generally achieved by operating athigher altitudes, using both aircraft and satellite imaging platforms.High-altitude wide-area imaging at high resolution quickly exceeds thecapacity of individual image sensors, so may utilize a scanning design.The scanning direction is typically perpendicular to the direction offlight, and the scanning mechanism may utilise a rotating mirror.

Scanning designs are generally not optimized for oblique imaging ormulti-view imaging in general.

SUMMARY OF THE INVENTION

In one aspect, the present invention provides a scanning camera forcapturing a set of images along a curved scan path within an area ofinterest, the scanning camera comprising an image sensor; a lens; ascanning mirror; and a drive coupled to the scanning mirror; wherein thedrive is operative to rotate the scanning mirror about a spin axisaccording to a spin angle; the spin axis is tilted relative to a cameraoptical axis; the scanning mirror is tilted relative to the cameraoptical axis and positioned to reflect an imaging beam into the lens;the lens is positioned to focus the imaging beam onto the image sensor;and the image sensor is operative to capture each image by sampling theimaging beam at a corresponding spin angle.

In another aspect, the present invention provides a scanning camerasystem comprising a first scanning camera facing in a first direction,and a second scanning camera according facing in a second directionsubstantially opposite to the first direction.

In another aspect, the present invention provides a scanning camera forcapturing a set of oblique images along a curved scan path within anarea of interest, the scanning camera comprising an image sensor; alens; a scanning mirror; and a drive coupled to the scanning mirror;wherein the drive is operative to rotate the scanning mirror about aspin axis according to a spin angle; the spin axis is tilted relative toa camera optical axis; the scanning mirror is tilted relative to thecamera optical axis and positioned to reflect an imaging beam into thelens; the camera optical axis is tilted at an oblique angle relative toan object plane within the area of interest; the lens is positioned tofocus the imaging beam onto the image sensor; and the image sensor isoperative to capture each image by sampling the imaging beam at acorresponding spin angle.

The spin axis of the scanning camera(s) may be substantially orthogonalto the camera optical axis.

The scanning mirror of the scanning camera(s) may be tilted atapproximately 45 degrees to the camera optical axis.

The set of images captured by (each of) the scanning camera(s) maycomprise at least some oblique images with substantially orthogonalviewing directions.

The set of images captured by (each of) the scanning camera(s) maycomprise at least one image with a substantially nadir viewing angle anda plurality of images with substantially oblique viewing angles.

The scanning camera(s) may comprise a correction mirror positioned tobend the camera optical axis between the lens and the scanning mirror.

The correction mirror may be tilted at approximately 45 degrees tocamera optical axis, thereby to bend the camera optical axis byapproximately 90 degrees.

The scanning camera(s) may comprise a correction mirror stage coupled tothe correction mirror, operative to rotate the correction mirror aboutat least one correction axis according to at least one correction angle.

In another aspect, the present invention provides a method of capturing,within an area of interest and using the scanning camera system, a firstset of images along a first curved scan path using the first scanningcamera, and a second set of images along a second curved scan path usingthe second scanning camera.

The set of images captured using each scanning camera may comprise atleast some oblique images with substantially orthogonal viewingdirections.

The set of images captured using each scanning camera may comprise atleast one image with a substantially nadir viewing angle and a pluralityof images with substantially oblique viewing angles.

In another aspect, the present invention provides a method for capturinga multi-view set of images of an area of interest, the multi-view set ofimages comprising, for each of a plurality of points within the area ofinterest, at least one nadir image and at least four oblique images fromfour substantially different viewing directions, the method comprisingmoving a dual-scan scanning camera along a survey path above the area ofinterest, and capturing, within selected intervals along the survey pathand using the dual-scan scanning camera, subsets of the multi-view setof images of the area of interest along pairs of opposed non-linear scanpaths.

The dual-scan scanning camera may comprise two scanning cameras facingin substantially opposite directions, the method comprising capturing,within each selected interval along the survey path and using eachscanning camera, a respective subset of the multi-view set of images ofthe area of interest along a respective non-linear scan path, each imagein the subset having a unique viewing angle and viewing direction pair.

The method may comprise, for each image within the subset, rotating ascanning mirror in an optical path of the corresponding scanning cameraabout a spin axis according to a spin angle, the spin axis tiltedrelative to a camera optical axis, the spin angle corresponding to aunique viewing angle and viewing direction pair.

The method may comprise, for each image within the subset, rotating ascanning mirror in an optical path of the corresponding scanning cameraabout a spin axis according to a spin angle, the spin axis aligned witha camera optical axis, and tilting the scanning mirror according to atilt angle, the spin angle and tilt angle pair corresponding to a uniqueviewing angle and viewing direction pair.

The method may comprise, for each image within the subset, rotating acamera assembly of the corresponding scanning camera about a spin axisaccording to a spin angle, the spin axis aligned with a camera opticalaxis, and tilting a tilting mirror in an optical path of the scanningcamera according to a tilt angle, the spin angle and tilt angle paircorresponding to a unique viewing angle and viewing direction pair.

The method may comprise, for each image within the subset, rotating acamera assembly of the corresponding scanning camera about a spin axisaccording to a spin angle, the spin axis substantially orthogonal to acamera optical axis, and tilting a tilting mirror in an optical path ofthe scanning camera according to a tilt angle, the spin angle and tiltangle pair corresponding to a unique viewing angle and viewing directionpair.

Each non-linear scan path may comprise a smooth curve.

Each non-linear scan path may comprise two linear segments, and the twolinear segments may be substantially orthogonal.

The two scanning cameras may share a single camera assembly.

The method may comprise multiplexing the single camera assembly betweenthe two scanning cameras by rotating a multiplexing mirror in theoptical paths of both scanning cameras between two operative positions.

In another aspect, the present invention provides a method forgenerating a 3D model of a surface, the method comprising moving adual-scan scanning camera along a survey path above the surface;capturing, at selected intervals along the survey path and using thedual-scan scanning camera, images of the surface along pairs of opposedcurved or shaped scan paths; inferring, using triangulation, 3Dpositions of common features among the images; and generating the 3Dmodel using the 3D positions.

In another aspect, the present invention provides a method forgenerating a true orthomosaic image of a surface, the method comprisingmoving a dual-scan scanning camera along a survey path above thesurface; capturing, at selected intervals along the survey path andusing the dual-scan scanning camera, images of the surface along pairsof opposed curved or shaped scan paths; inferring, using triangulation,3D positions of common features among the images; generating a 3D modelof the surface using the 3D positions; and projecting the 3D modelaccording to a viewing direction to generate the true orthomosaic image.

DRAWINGS—FIGURES

FIG. 1A shows a right side elevation of a scanning camera.

FIG. 1B shows a perspective view of the scanning camera.

FIG. 1C shows the scanning camera with a housing.

FIG. 2 shows a parts explosion of the scanning camera.

FIG. 3 shows a block diagram of the scanning camera.

FIG. 4A shows the relationship between the image sensor and the sensorfield.

FIG. 4B shows the structure of a sensor point beam.

FIG. 4C shows the structure of the imaging beam.

FIG. 5A shows a right side elevation of the scanning camera with itsscanning mirror at a zero spin angle.

FIG. 5B shows a right-front elevation of the scanning camera with itsscanning mirror at a zero spin angle.

FIG. 5C shows a front elevation of the scanning camera with its scanningmirror at a zero spin angle.

FIG. 5D shows a right side elevation of the scanning camera with itsscanning mirror at an extreme spin angle.

FIG. 5E shows a right-front elevation of the scanning camera with itsscanning mirror at an extreme spin angle.

FIG. 5F shows a front elevation of the scanning camera with its scanningmirror at an extreme spin angle.

FIG. 6 shows the scanning camera imaging geometry for a single scanposition.

FIG. 7A shows a scan field of the scanning camera.

FIG. 7B shows the scan field of the scanning camera from a higheraltitude.

FIG. 8A shows the intersection of corner sensor point beams with thescanning mirror of the scanning camera at an extreme spin angle.

FIG. 8B shows the intersection of corner sensor point beams with thescanning mirror of the scanning camera through a full scan range.

FIG. 9A shows a flight path of a survey aircraft during one pass of anaerial survey.

FIG. 9B shows a flight path of a survey aircraft during two orthogonalpasses of an aerial survey.

FIG. 10 shows a block diagram of an aerial imaging system incorporatingthe scanning camera.

FIG. 11 shows an activity diagram for an aerial survey control algorithmand scanning camera control algorithm.

FIG. 12A shows a dual-scan scanning camera.

FIG. 12B shows the dual-scan scanning camera with a housing.

FIG. 13 shows a parts explosion of the dual-scan scanning camera.

FIG. 14A shows a top plan view of the dual-scan scanning camera.

FIG. 14B shows a bottom plan view of the dual-scan scanning camera.

FIG. 14C shows a top plan view of the dual-scan scanning camera with itsscanning mirror at multiple spin angles.

FIG. 14D shows a bottom plan view of the dual-scan scanning camera withits scanning mirror at multiple spin angles.

FIG. 15 shows the dual-scan scanning camera mounted on an AMC platform.

FIG. 16 shows a parts explosion of the mounted dual-scan scanningcamera.

FIG. 17A shows a scan field of the dual-scan scanning camera.

FIG. 17B shows a scan field of the dual-scan scanning camera with a45-degree heading.

FIG. 18A shows two successive scan fields of the dual-scan scanningcamera.

FIG. 18B shows two adjacent scan fields of the dual-scan scanning camerafrom adjacent flightlines.

FIG. 19 shows the scan field of the dual-scan scanning camera relativeto a multi-line survey path.

FIG. 20A shows an elevation of the scan field of the dual-scan scanningcamera.

FIG. 20B shows an elevation of the overlapping scan fields of thedual-scan scanning camera from three adjacent flightlines.

FIG. 21 shows the scan field of the dual-scan scanning camera carried bya survey aircraft.

FIG. 22A shows a block diagram of the dual-scan scanning camera.

FIG. 22B shows a block diagram of a triple-scan scanning camera.

FIG. 23A shows a scan field of the triple-scan scanning camera.

FIG. 23B shows a scan field of the triple-scan scanning camera with a45-degree heading.

FIG. 24 shows a tabulation of the performance of the dual-scan scanningcamera at different altitudes for a fixed GSD.

FIG. 25 shows an activity diagram for a photogrammetry process for 3Dsurface reconstruction.

FIG. 26A shows the scanning camera tilted for oblique imaging.

FIG. 26B shows the scanning camera and its scanning mirror both tiltedfor oblique imaging.

FIG. 26C shows the scanning camera with a single mirror, tilted foroblique imaging.

FIG. 27 shows a wide scan field of a dual-scan oblique scanning camera.

FIG. 28A shows a narrow scan field of a triple-scan oblique scanningcamera.

FIG. 28B shows two overlapping scan fields of the triple-scan obliquescanning camera from two orthogonal passes of an aerial survey.

FIG. 29A shows two adjacent scan fields of a dual-scan oblique scanningcamera from adjacent flightlines.

FIG. 29B shows four scan fields of the dual-scan oblique scanning camerafrom adjacent flightlines within two orthogonal passes of an aerialsurvey.

FIG. 30A shows the intersection of corner sensor point beams with thescanning mirror of the oblique scanning camera at an extreme spin angle.

FIG. 30B shows the intersection of corner sensor point beams with thescanning mirror of the oblique scanning camera through a full scanrange.

FIG. 31 shows the scan field of the triple-scan oblique scanning cameracarried by a survey aircraft.

FIG. 32A shows a perspective view of a linear scanning camera.

FIG. 32B shows a scan field of the linear scanning camera.

FIG. 32C shows a scan field of the linear scanning camera with the imagesensor rotated.

FIG. 33A shows a scan field of the dual-scan scanning camera using alarger rectangular image sensor.

FIG. 33B shows a crossed scan field of a dual-scan linear scanningcamera using the rectangular image sensor.

FIG. 34A shows a linear scanning camera with a spinning mirror.

FIG. 34B shows a steerable scanning camera with a spinning mirror.

FIG. 34C shows a linear scanning camera with a spinning camera assembly.

FIG. 34D shows a steerable scanning camera with a spinning cameraassembly.

FIG. 34E shows a linear scanning camera with a swinging camera assembly.

FIG. 34F shows a steerable scanning camera with a swinging cameraassembly.

FIG. 35A shows a shaped scan field of a steerable scanning camera.

FIG. 35B shows a shaped scan field of a dual-scan steerable scanningcamera.

FIG. 36 lists the equations governing the viewing angle and viewingdirection of a steerable scanning camera.

FIG. 37 shows the shaped scan field of a dual-scan steerable scanningcamera carried by a survey aircraft.

FIG. 38A shows a multiplexed scanning camera with a spinningmultiplexing mirror.

FIG. 38B shows a multiplexed scanning camera with a tilting multiplexingmirror.

FIG. 38C shows a multiplexed oblique scanning camera with a tiltingmultiplexing mirror.

FIG. 38D shows a multiplexed steerable scanning camera with a tiltingmultiplexing mirror.

DRAWINGS—REFERENCE NUMERALS

-   100 Scanning camera.-   102 Camera.-   104 Lens assembly.-   106 Camera optical axis.-   108 Scanning optical axis.-   110 Correction mirror stage.-   112 Correction mirror.-   114 Correction mirror housing.-   120 Scanning mirror drive.-   122 Scanning mirror.-   124 Scanning mirror mount.-   126 Scanning mirror spin axis.-   128 Scanning mirror spin angle.-   130 Mount plate.-   132 Main housing.-   134 Correction mirror assembly.-   136 Scanning mirror assembly.-   140 Scanning camera controller.-   142 Camera controller.-   144 Image sensor.-   146 Lens controller.-   148 Focusable lens.-   150 Scanning camera control & data.-   152 Image data.-   160 Imaging beam.-   162 Sensor field.-   164 Chief ray.-   166 Aperture.-   168 Sensor point.-   170 Sensor point beam.-   172 Sensor field point.-   174 Sensor point beam cross-section.-   180 Scan path.-   182 Viewing angle.-   184 Viewing direction.-   200 Flight management system computer.-   202 Pilot user interface.-   204 Autopilot.-   206 Photo storage.-   208 GNSS receiver.-   210 IMU.-   212 AMC platform.-   220 Aerial survey control.-   222 Wait for start of next flightline.-   224 Wait for next capture position.-   226 Send scan start signal.-   230 Scanning camera control.-   232 Wait for scan start signal.-   234 Capture image.-   236 Rotate scanning mirror.-   238 Reset scanning mirror.-   300 Dual-scan scanning camera.-   302 Dual-scan mount plate.-   304 Mount rod.-   306 Dual-scan main housing.-   400 Survey aircraft.-   402 Survey aircraft heading.-   404 Survey path.-   406 Second-pass survey path.-   500 Triple-scan scanning camera.-   600 Scan field.-   610 Dual scan field.-   620 Nadir sub-field.-   622 Oblique sub-field.-   630 Nadir scan field.-   640 Oblique scan field.-   650 Dual oblique scan field.-   700 Camera mount.-   702 Vibration isolator.-   800 Photos.-   802 GNSS positions.-   804 IMU orientations.-   806 Scan directions.-   808 Estimate photo positions & orientations.-   810 Positions & orientations.-   812 Reconstruct 3D surface.-   814 Textured 3D surface.-   900 Linear scanning camera.-   902 Camera assembly.-   904 Scanning camera drive.-   906 Scanning camera spin axis.-   908 Fixed mirror.-   910 Linear scan field.-   912 Crossed linear scan field.-   920 Steerable scanning camera.-   922 Tilting mirror.-   924 Tilting mirror drive.-   926 Tilted imaging beam.-   930 Dual-scan steerable scanning camera.-   940 Shaped scan field.-   950 Dual shaped scan field.-   960 Multiplexing mirror.-   962 Multiplexing mirror drive.

DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS

FIGS. 1A and 1B show an embodiment of a scanning camera 100 according toone aspect of the present invention. The scanning camera 100 comprises acamera 102, lens assembly 104, correction mirror 112, scanning mirrordrive 120, and scanning mirror 122. The scanning mirror 122 is coupledto the scanning mirror drive 120 via a scanning mirror mount 124. Thescanning camera module 100 is configured to scan an imaging beam 160across an area of interest by rotating the scanning mirror 122 about aspin axis 126. The rotation is defined by a spin angle 128.

The optical axis of the scanning camera 100, i.e. the axis of theimaging beam 160, is referred to as the camera optical axis 106 betweenthe lens assembly 104 and the scanning mirror 122, and as the scanningoptical axis 108 between the scanning mirror 122 and the object planewithin the area of interest.

FIG. 1C shows the scanning camera 100 with a mount plate 130 supportingthe lens assembly 104 and the scanning mirror drive 120, and aprotective housing 132. Cutaways in the lower edge of the housing 132accommodate the imaging beam 160 at extreme spin angles.

FIG. 2 shows a parts explosion of the scanning camera 100.

FIG. 3 shows a block diagram of the scanning camera 100. Open arrowheadsdenote control & data interconnects. Solid arrowheads denote mechanicalcouplings. Large arrows denote optical flow. The camera 102 comprises acamera controller 142 and an image sensor 144 controlled by the cameracontroller 142. The lens assembly 104 optionally comprises a lenscontroller 146 and an electronically focusable lens 148 controlled bythe lens controller 146. The lens 148 may alternatively be manually orfactory focused (and the lens controller 146 omitted). An optionalcorrection mirror stage 110 (not shown in FIG. 1A or 1B) effectsrotation of correction mirror 112 about one or more axes to effectforward motion correction (FMC) and/or angular motion correction (AMC)of the scanning camera 100. The correction mirror 112 may alternativelybe fixed (and the correction mirror stage 110 omitted). The scanningmirror drive 120 effects rotation of the scanning mirror 122 about thespin axis 126. A scanning camera controller 140 controls the cameracontroller 142, lens controller 146, correction mirror stage 110 andscanning mirror drive 120. The scanning camera controller 140 isresponsive to instructions from a controlling system via control & datainterconnect 150. It returns image data captured by the image sensor144, and associated data, to the controlling system via interconnect150. Alternatively, the camera controller 142 may return image data viaa separate interconnect 152.

When the correction mirror 112 is fixed its primary purpose is to bendthe camera optical axis 106 by 90 degrees to allow the lens assembly 104(and camera 102) to be conveniently mounted vertically rather thanhorizontally. If this mounting option is not required then thecorrection mirror 112 may be omitted, with the scanning mirror thenreflecting the imaging beam 160 directly into the lens assembly 104rather than indirectly via the correction mirror 112. An example of thisis discussed later in this specification with reference to FIG. 26C.

FIG. 4A shows the relationship between the image sensor 144 and itscorresponding sensor field 162 in the object plane (which typicallycoincides with the ground). The extent of the sensor field 162 isdefined by the four chief rays 164 from the corners of the image sensor144 through the center of the aperture 166 of the lens 148. FIG. 4Bshows, for a single illustrative point 168 on the image sensor 144, thecorresponding sensor point beam 170. The sensor point beam 170 comprisesall rays from the sensor point 168 through the aperture 166 and focusedat a corresponding field point 172 in the object plane.

FIG. 4C shows the structure of the imaging beam 160, which comprises thesum of all sensor point beams 170 from all points 168 on the imagesensor 144. The cross-section of the imaging beam 160 isimage-sensor-shaped (i.e. rectangular) at the image sensor 144 and atthe sensor field 162, aperture-shaped (e.g. polygonal or circular) atthe aperture 166, and an intermediate shape at intermediate points alongthe imaging beam 160.

Referring to the coordinate axes shown in FIGS. 1A and 1B, thelongitudinal axis of the scanning camera 100 is the x axis, the lateralaxis is the y axis, and the vertical axis is the z axis. In oneexemplary configuration of the scanning camera 100, as shown in FIGS. 1Aand 1B, and FIGS. 5A through 5C, the spin axis 126 is vertical; the lensassembly 104 is mounted vertically and pointing down; the scanningmirror 122 is mounted at 45 degrees to the spin axis 126, facingbackward in the direction of the correction mirror 112 when the spinangle 128 is zero (defined relative to the negative x axis); and thecorrection mirror 112 is mounted at 45 degrees to the camera opticalaxis 106, immediately below the lens assembly 104. Thus, the verticalcamera optical axis 106 is reflected horizontally by the correctionmirror 112, and, when the spin angle 128 is zero, is reflectedvertically again by the scanning mirror 122 to become the (nadir)scanning optical axis 108. In the opposite direction, i.e. for incominglight, a vertical (nadir) imaging beam 160 is reflected horizontally bythe scanning mirror 122 onto the correction mirror 112, and is reflectedvertically again by the correction mirror 112 into the lens assembly104.

The scanning mirror drive 120 is configured to rotate the scanningmirror 122 about the spin axis 126. As shown in FIGS. 5D through 5F,when the scanning mirror 122 is rotated away from facing the correctionmirror 112, i.e. with a positive spin angle 128, the imaging beam 160 isdeflected both laterally and longitudinally.

A vertically-oriented spin axis 126 differs from prior-art scanningcameras where the spin axis is typically horizontally-oriented to effectlinear side-to-side scanning. See for example Patel (U.S. Pat. No.5,550,669, “Flexure design for a fast steering scanning mirror).

FIG. 6 shows the imaging geometry of the scanning camera 100. For aparticular spin angle 128, the scanning optical axis 108 (i.e. thepointing direction of the imaging beam 160) is defined by a viewingangle 182 relative to the negative z axis, and a viewing direction 184relative to the positive or negative y axis (depending on the sign ofthe spin angle 128). The position and orientation of the sensor field162 on the ground is therefore likewise determined. As the spin angle128 of the scanning mirror 122 is varied from a negative extreme to apositive extreme, the scanning optical axis 108 traces out a curved scanpath 180 on the ground.

Scanning cameras are typically designed to trace out a linear scan path,i.e. where the viewing direction is fixed throughout the scan (exceptwhere it switches direction as the scan passes through the nadir point).However, a curved scan path 180 has specific advantages that arediscussed later in this specification.

The curved scan path 180 may be flattened by changing the fixed tilt ofthe spin axis 126, i.e. away from a purely vertical orientation towardsa horizontal orientation. As the spin axis 126 approaches a purelyhorizontal orientation the scan path 180 becomes increasingly straight.

The viewing angle 182 at zero spin can be varied by changing the angleat which the scanning mirror 122 is mounted relative to the spin axis126.

For the illustrative configuration of the scanning camera 100 describedabove, where both the correction mirror 112 and scanning mirror 122 aretilted at 45 degrees to the z axis, the viewing angle 182 equals theabsolute value of the spin angle 128, and the viewing direction 184equals the spin angle 128.

At a spin angle 128 of 45 degrees, the viewing angle 182 and the viewingdirection 184 are therefore both conveniently 45 degrees, which is idealfor oblique imaging. If the spin angle is varied from minus 45 degreesto plus 45 degrees, the viewing angle varies from 45 degrees to zero to45 degrees, and the viewing direction varies from minus 45 degrees toplus 45 degrees.

Commercial uses of oblique imagery typically desire a viewing angle 182of 40 to 45 degrees, although any significantly off-nadir viewing angle182, even if less than 40 degrees, may be termed oblique.

Since the spin of the scanning mirror 122 is used to effect scanning ofthe imaging beam 160, the spin range of the scanning mirror 122 isreferred as the scan range of the scanning camera 100.

In one illustrative configuration of the scanning camera 100, the imagesensor 144 is an ON Semiconductor PYTHON 25K with 5120×5120 square 4.5um pixels, the lens 148 has a focal length of 300 mm, and the scan rangeis a symmetric 90 degrees (i.e. corresponding to a spin angle of minus45 degrees through plus 45 degrees).

FIG. 7A shows a scan field 600 of the illustrative scanning camera 100operated at an altitude of 14,000 feet. It consists of 27 overlappingsensor fields 162 on the ground. The grid spacing in FIG. 7A andthroughout the figures is 2.5 km. The longitudinal axis of the scanningcamera 100 is aligned with the survey aircraft heading 402, and thescanning camera 100 is facing forward.

FIG. 7B shows a scan field 600 of the illustrative scanning camera 100operated at an altitude of 28,000 feet. It consists of 55 overlappingsensor fields 162 on the ground.

The required size and shape of the scanning mirror 122 is determined bythe intersection of the imaging beam 160 with the face of the scanningmirror 122. This in turn is determined by the maximum spin anglesupported by a particular scanning camera 100, and the size and shape ofthe imaging beam 160. As discussed in relation to FIGS. 4A through 4C,the imaging beam 160 comprises the sum of all sensor point beams 170from all points 168 on the image sensor 144. It is thus determined bythe size of the image sensor 144, the focal length of the lens 148, andthe diameter of the aperture 166. It is also affected by the spacing ofthe aperture 166, correction mirror 112, and scanning mirror 122. Theillustrative scanning camera 100 has an aperture 166 with a diameter of75 mm (i.e. it has an f-number of f/4).

Although the imaging beam 160 comprises the sum of all sensor pointbeams 170, its maximum width is determined by the four sensor pointbeams 170 from the four corners of the image sensor 144. For analysispurposes the imaging beam 160 may therefore be represented by those fourcorner sensor point beams 170. And although the scanning mirror 122 mustreflect the imaging beam 160 throughout the scan range, its maximumwidth is determined by the imaging beams 160 at the two extremes of thescan range.

FIG. 8A shows the intersection 174 of each of the four corner sensorpoint beams 170 with the scanning mirror 122 at the maximum spin angleof 45 degrees, determined by ray tracing. FIG. 8B shows the intersection174 of each of the four corner sensor point beams 170 with the scanningmirror 122 throughout the full scan range. The figures illustrate howthe required width of the mirror is determined by the sensor point beams170 at the extreme ends of the scan range, and the required height ofthe mirror is almost uniformly determined by sensor point beams 170throughout the scan range. Note that the shape of the scanning mirror122 is symmetric horizontally about the spin axis 126, but is offsetdownwards relative to the spin axis.

Although illustrated with reference to a set of specific parameters, themechanical design of the scanning camera 100 can be adapted to anydesired set of parameters without affecting its intrinsic design. Thisincludes a larger or smaller image sensor, focal length, aperture, andscan range.

An aerial camera has a finite capture field, such as the scan field 600of the scanning camera 100. The usual aim of an aerial survey is tofully capture images of the ground within a chosen survey area, and thisis normally accomplished by flying a survey aircraft 400 along aserpentine path 404 consisting of a sequence of parallel flightlineslinked by turns, as shown in FIG. 9A. The flightline spacing is chosento ensure overlap between the capture fields of adjacent flightlines.

When certain views, such as oblique views, are best captured with theaircraft flying in a particular direction, a survey may be accomplishedby flying two orthogonal passes of the survey area, as shown in FIG. 9B.

FIG. 10 shows a block diagram of an aerial imaging system incorporatingthe scanning camera 100. A flight management system (FMS) computer 200runs FMS software that manages the flight of the survey aircraft 400along a survey path 404, and controls the operation of the scanningcamera 100 to ensure capture of the survey area. Based on a storedsurvey plan, the FMS may issue guidance to a pilot via a pilot userinterface 202, and/or may automatically guide the aircraft via anautopilot 204. The FMS receives 3D position data from a GlobalNavigation Satellite System (GNSS) receiver 208, and 3D orientation datafrom an Inertial Measurement Unit (IMU) 210. The system optionallyincludes an Angular Motion Correction (AMC) platform 212 that correctsfor angular motion of the survey aircraft in one or more dimensions. TheAMC platform 212 is responsive to orientation changes reported by theIMU 210. The scanning camera 100 is mounted on the AMC platform 212 toensure its imaging beam 160 points in a consistent direction, to avoidgaps between successive (and adjacent) scan fields, while minimizing therequired nominal overlap. The IMU 210 may be mounted on the AMC platform212 so that it directly reports the orientation of the scanning camera100. If the IMU 210 is not mounted on the AMC platform 212 then theorientation of the scanning camera 100 may be inferred by the FMS fromthe orientation reported by the IMU 210 and any angular correctionreported by the AMC platform 212. The FMS receives photos and associateddata from the scanning camera 100 which it stores in attached photostorage 206.

Angular motion correction may be provided by the AMC platform 212 andthe correction mirror 112 in unison. For example, the AMC platform 212may provide yaw correction and the correction mirror 112 may providepitch and roll correction; or they may each provide partial pitch androll correction; or the correction mirror 112 may be used only forcorrecting for forward movement of the aircraft (i.e. FMC), and/or formovement of the scanning mirror 122, during the exposure period of theimage sensor 144. FMC, if required, may alternatively be provided by anysuitable mechanism, including time delay integration (TDI) in the imagesensor 144, or compensating movement of the image sensor.

FIG. 11 shows an activity diagram for an aerial survey control process220 and scanning camera control process 230 that may be run on the FMScomputer 200 and scanning camera controller 140 respectively.

The aerial survey control process 220 controls capture of images alongeach flightline in the stored survey plan. At the start of eachflightline, either at the start of the survey or after a turnaroundbetween flightlines, it waits (at step 222) until the aircraft is at thestart of the flightline. For each capture position along the flightlinethe process waits (at step 224) until the aircraft is at the captureposition, sends (at step 226) a scan start signal to the scanning cameracontrol process 230, and then repeats. At the end of a flightline theprocess returns to waiting for the start of the next flightline. At theend of the last flightline the process exits.

The survey control process 220 calculates each capture position based onthe previous capture position, the longitudinal extent of the scanfield, and the desired longitudinal overlap between successive scanfields. It may also take into account the orientation of the aircraftreported by the IMU 210, if this is not fully corrected by AMC, toensure overlap between successive scan fields. It monitors the positionof the aircraft via the GNSS receiver 208.

The scanning camera control process 230 waits (at step 232) for a scanstart signal from the aerial survey control process 220, and thencontrols capture of images at each spin angle within the scan. At eachspin angle 128 it captures an image (at step 234) via the image sensor144 via the camera controller 142, rotates the scanning mirror 122 (atstep 236) to the next spin angle via the scanning mirror drive 120, andthen repeats. At the end of the scan, i.e. when the scan range isexhausted, the process optionally resets the scanning mirror 122 (atstep 238) to its starting spin angle, and then returns to waiting forthe next scan start signal. To avoid having to reset the scanning mirrorat the end of each scan, the process may alternatively scanbidirectionally, i.e. in one direction on even-numbered scans and in theopposite direction on odd-numbered scans.

The scanning camera controller 140 may monitor the focus and exposure ofcaptured images and adjust the focus and aperture of the focusable lens148 via the lens controller 146 to compensate for any deviation fromideal focus or exposure. It may also adjust image exposure by adjustingthe exposure time of the image sensor 144. It may also use exposurebracketing to capture imagery with a wider dynamic range.

The distribution of control functions within the scanning camera 100 isdescribed in the foregoing with reference to a possible embodiment, butit should be clear that control functions could be distributeddifferently across the FMS computer 200, scanning camera controller 140,camera controller 142, and lens controller 146 to achieve the sameeffect. For example, a controller may be omitted and its functionsperformed by another of the controllers or the computer.

The camera controller 142, if returning image data via a separateinterconnect 152, may utilize a high-speed communication standard suchas CoaXPress (CXP).

The camera 102 may be a commercial off-the-shelf (COTS) machine visioncamera that incorporates the desired high-speed image sensor 144. Forthe illustrative PYTHON 25K image sensor 144, the camera 102 may, forexample, be an Adimec S-25A80 which supports output at the full framerate of the PYTHON 25K image sensor 144 over a CXP interconnect 152.

Although the illustrative image sensor 144 is an RGB image sensor,monochrome, near infrared and multi-spectral image sensing may also beutilized.

The lens assembly 104 may be a COTS lens, such as a high-performancedioptric (refractive) prime lens incorporating multiple lens elementsand providing motorized focus and aperture adjustment. For longer focallengths a catoptric (reflective) telescope lens may be used, or a hybridcatadioptric lens.

The correction mirror stage 110 may be any suitable tilt or tip-tiltstage that provides a sufficient angle range for AMC or FMC. It may, forexample, comprise one or more piezo-electric actuators with associatedcontrol (or control may be incorporated in the scanning cameracontroller 140). It may incorporate one or more position sensors forclosed-loop control.

The scanning mirror drive 120 may be any suitable rotary drive thatprovides sufficient torque to rotate the scanning mirror 122 from onespin angle 128 to the next during the available time interval betweensuccessive shots. It may, for example, comprise a stepper motor orpiezo-electric actuator or motor with associated control (or control maybe incorporated in the scanning camera controller 140). It mayincorporate a position sensor for closed-loop control.

A linear actuator may be suitably coupled to provide rotationalmovement, e.g. via a rack and pinion mechanism. For examples ofpiezo-electric actuators and rotary couplings see Johansson et al. (U.S.Pat. No. 6,337,532, “Fine walking actuator”), Johansson (U.S. Pat. No.7,420,321, “Heat efficient micromotor”), and Bexell et al. (U.S. Pat.No. 9,293,685, “Rotating load bearer”), the contents of all of which areherein included by cross reference.

FIG. 12A shows an embodiment of a dual-scan scanning camera 300. Thedual-scan scanning camera 300 comprises two scanning cameras 100, onefacing forward and the other facing backward. Reference numerals forcomponents associated with each scanning camera 100 are suffixed “f”(for forward) or “b” (for backward) as appropriate.

FIG. 12B shows the dual-scan scanning camera 300 with a mount plate 302supporting the lens assemblies 104 and the scanning mirror drives 120,and a protective housing 306. Cutaways in the lower edge of the housing306 accommodate the imaging beams 160 at extreme spin angles. The mountplate 302 incorporates a mount rod 304 for attaching the dual-scanscanning camera 300 to a camera mount in a vertically-adjustablefashion.

FIG. 13 shows a parts explosion of the dual-scan scanning camera 300,including correction mirror assemblies 134, each comprising a correctionmirror 112 and correction mirror housing 114, and scanning mirrorassemblies 136, each comprising a scanning mirror 122 and scanningmirror mount 124.

FIG. 14A and FIG. 14B show top and bottom plan views respectively of thedual-scan scanning camera 300.

FIG. 14C and FIG. 14D show top and bottom plan views respectively of thedual-scan scanning camera 300 with its scanning mirrors 122 f and 122 bat multiple spin angles, illustrating clearance during operation betweenthe adjacent scanning mirrors 122 f and 122 b and between the adjacentimaging beams 160 f and 160 b.

The dual-scan scanning camera 300 may be mounted over a camera hole inthe floor of a survey aircraft or spacecraft, or in the floor of anexternal pod carried by a survey aircraft or spacecraft. The camera holemay incorporate an optical-grade glass window, e.g. if the aircraft ispressurized.

FIG. 15 shows the dual-scan scanning camera 300 mounted, via cameramount 700, on an AMC platform 212. The scanning camera controllers 140and IMU 210 are also shown mounted on the camera mount 700. Thedual-scan scanning camera 300 attaches to the camera mount 700 via itsmount rod 304, which allows its vertical position relative to the mountto be adjusted according to the depth of the camera hole above which itis mounted, i.e. to ensure that the imaging beams 160 have sufficientclearance throughout their scan range, subject also to the maximumanticipated aircraft angular motion. The mount rod 304 may be attachedto the camera mount 700 using a mount bolt (not shown) passing laterallythrough the camera mount and mount rod. The mount rod may incorporate aseries of holes (not shown) along its length that accept the bolt, theuse of one of which sets the vertical position of the scanning camera300.

FIG. 16 shows a parts explosion of the mounted dual-scan scanning camera300. The camera mount 700 mounts to the AMC platform 212 via a set ofvibration isolators 702. These may be of any suitable type, includingwire rope isolators and elastomeric isolators.

The AMC platform 212 may be any suitable one-, two- or three-axis AMCplatform. It may, for example, be a three-axis Lead′Air SteadyTrackSTX-550.

Aerial imagery may be utilised in a number of different ways. Aerialphotos may be used individually, or may be orthorectified and stitchedinto continuous mosaics. They may also be used to reconstruct the 3Dshape of the ground, and the resulting 3D model may be textured with theimagery.

The use of two scanning cameras 100, facing in opposite directions,allows the dual-scan scanning camera 300 to capture oblique views infour directions spaced approximately 90 degrees apart. This supportstraditional uses of oblique aerial photos, as well as robust 3D surfacereconstruction.

FIG. 17A shows the dual scan field 610 of the dual-scan scanning camera300 operated at an altitude of 14,000 feet with a northerly aircraftheading 402. The dual scan field 610 comprises a forward scan field 600f and a backward scan field 600 b. FIG. 17B shows the same dual scanfield 610 but with a 45-degree (north-east) aircraft heading 402.

Using a 45-degree heading 402 ensures that the four oblique viewscaptured by the dual scan field 610 are aligned with the four cardinaldirections, thus satisfying market expectations for oblique imagery.

FIG. 18A shows two successive dual scan fields 610 a and 610 b of thedual-scan scanning camera 300 along a single-flightline survey path 404,illustrating longitudinal overlap between successive dual scan fields610.

FIG. 18B shows two adjacent dual scan fields 610 a and 610 b of thedual-scan scanning camera 300 from adjacent flightlines within atwo-flightline survey path 404, illustrating lateral overlap betweenadjacent dual scan fields 610.

FIG. 19 shows the dual scan field 610 of the dual-scan scanning camera300 relative to a seven-flightline survey path 404.

FIG. 20A shows an elevation of the dual scan field 610 of the dual-scanscanning camera 300, representing the indicated east-west cross-sectionof the dual scan field 610 in FIG. 19 (but note that because each scanfield 600 is curved, the actual sensor fields overlapping thecross-section come from multiple successive scan fields). FIG. 20A showsthe dual scan field 610 segmented into a nadir sub-field 620 and leftand right oblique sub-fields 622 a and 622 b. These sub-fields representthe optimal nadir and oblique contributions of the dual scan field 610assuming the flightline spacing of FIG. 19. This is further illustratedin FIG. 20B, which shows dual scan fields 610 from three adjacentflightlines, where each oblique sub-field 622 is shown to be delineatedby the edge of the dual scan field 610 from the adjacent flightline. Theleft and right oblique sub-fields 622 a and 622 b are associated withthe backward scan field 600 b and forward scan field 600 f respectively,while the nadir sub-field 620 is associated with both scan fields (i.e.each half of the nadir sub-field is associated with a respective scanfield). Likewise, the corresponding left and right oblique sub-fields inthe orthogonal north-south direction of the scan field 610 in FIG. 19are associated with the forward scan field 600 f and backward scan field600 b respectively, and the north-south nadir sub-field 620 isassociated with both.

Segmentation into a nadir sub-field 620 and left and right obliquesub-fields 622 is relevant to traditional uses of oblique and nadiraerial imagery. When reconstructing 3D surfaces, imagery from theentirety of each dual scan field 610 may be used.

FIG. 21 shows a perspective view of the dual scan field 610 of thedual-scan scanning camera 300 carried by a survey aircraft 400.

Although the figures show the dual scan field 610 comprising twocomplete scan fields 600 f and 600 b, the nadir portion of one or theother scan field may be omitted for efficiency since they overlapsubstantially. If the nadir portion is omitted from one scan field oneven scans and the other scan field on odd scans then the average shotrate of both scans can be reduced. Alternatively, half of the nadirportion of one scan field and half of the nadir portion of the otherscan field may be omitted to the same effect, this equating to omittingthe capture of one composite nadir sub-field 620 (i.e. one of the twoorthogonal east-west and north-south nadir sub-fields 620).

Although the survey platform is illustrated as a manned fixed-wingaircraft 400, the survey platform may be any suitable moving platform,including an unmanned aircraft, a rotary-wing aircraft, an orbitingsatellite, a spacecraft, etc.

If the survey platform is an aircraft then the survey path 404 istypically serpentine, as shown in FIG. 19. If the survey platform is anorbital satellite then the survey path 404 typically consists of a setof parallel orbital tracks, with the survey platform moving in the samedirection along each track. However, any suitable survey path 404 may beused so long as overlap between scan fields 610 is maintained to ensurefull coverage of the area of interest.

For simplicity the figures show each scan field as if captured at aninstant in time, and hence corresponding to a specific point on thesurvey path 404. In practice each scan field takes a finite time tocapture, and so is associated with a time interval rather than a timeinstant, and an interval on the survey path rather than a point. Inaddition, the multiple scanning cameras 100 of a multi-scan scanningcamera, such as the dual-scan scanning camera 300, need not be fullysynchronized, but typically overlap sufficiently in time that they canbe associated pair-wise (etc.) with a set of overlapping intervals alongthe survey path.

FIG. 22A shows a block diagram of the dual-scan scanning camera 300,comprising its two scanning cameras 100 f and 100 b for forward andbackward capture respectively.

As is evident from FIG. 7A, as the magnitude of the spin angle 128increases away from zero at the center of the scan field 600, the sizeof the sensor field 162 and hence the ground sampling distance (GSD)both increase accordingly. This results in the capture of obliqueimagery with a lower spatial resolution than the nadir imagery withinthe same scan field 600. If a third scanning camera 100 (or any suitableaerial camera) is used to capture the nadir imagery, then the GSD of theoblique imagery can be decoupled from the GSD of the nadir imagery,either by using a lens 148 with a longer focal length for the scanningcameras 100 used to capture the oblique imagery, or an image sensor 144with smaller pixel pitch.

FIG. 22B shows a block diagram of a triple-scan scanning camera 500,comprising three scanning cameras 100, comprising one scanning camera100 n for nadir capture and two scanning cameras 100 f and 100 b forforward and backward oblique capture respectively.

FIG. 23A shows a scan field 610 of the triple-scan scanning camera 500.For illustrative purposes, the focal length of the oblique scanningcameras 100 f and 100 b is 400 mm, while the focal length of the nadirscanning camera 100 n remains at 300 mm. The oblique scan fields 600 areshown as partial, i.e. with the nadir portions of each scan omitted.Likewise, the nadir scan field 630 is shown with a limited scan rangecovering only the required nadir portion. Alternatively, the obliquescan fields and/or the nadir scan field may be full scan fields,yielding imagery with more redundancy (and multiple resolutions).However, the use of partial scan fields allows a higher scan rate thanthe use of full scan fields.

FIG. 23B shows the scan field 610 of the triple-scan scanning camerawith a 45-degree survey aircraft heading 402.

More than two scanning cameras 100 may also be used, suitably rotated(e.g. to ensure evenly-spaced viewing directions), to capture additionalviews.

Although the figures show the dual scan field 610 comprising a forwardscan field 600 f and backward scan field 600 b captured using forward-and backward-facing scanning cameras 100, the scan field may insteadcomprise left and right scan fields 600 captured using a left- andright-facing scanning cameras 100. The choice of facing direction may beinfluenced by the shape of the available camera hole in a surveyaircraft.

FIG. 24 shows a tabulation of the performance of the dual-scan scanningcamera 300 at different altitudes for a fixed GSD. The altitude isvaried from 2000 feet to 50,000 feet in 2000-feet increments, with theaircraft speed increasing in steps to reflect realistic aircraftchoices. The GSD is fixed at 5 cm, and the focal length is calculated toyield the fixed GSD (although in practice the focal lengths of availableCOTS lenses is more constrained). The scan rate indicates the number ofscans per second, for each scanning camera 100 within the dual-scanscanning camera 300. The shot count indicates the number of shots withineach scan field 600. The shot rate indicates the number of shots persecond within a scan. The line spacing indicates the spacing offlightlines based on a 30-degree flightline spacing. The capture rateindicates the overall productivity of the dual-scan scanning camera 300in terms of area captured per hour.

The allowable shot rate is bounded by the maximum frame rate of theimage sensor 144. The illustrative PYTHON 25K image sensor 144 has amaximum frame rate of 80 fps, allowing 5 cm imaging at 400 knots up toan altitude of 38,000 feet. Higher-rate image sensors may be used forhigher-altitude operation, or lower-rate image sensors may bemultiplexed.

In general, any number of scanning cameras 100 may be deployed, suitablyrotated (e.g. to ensure evenly-spaced viewing directions), to capture adesired number of views.

Any number of scanning cameras 100 may also be deployed, facing in thesame direction, to increase capture throughput. For example, if thenumber of scanning cameras facing in a particular direction is doubled,then the effective scan rate in that direction is also doubled. If thecapture rate is limited by the frame rate of the image sensor 144 or bythe mechanical movement of the scanning mirror 122, then the use ofmultiple scanning cameras 100 can be used to overcome those limits.Increased numbers of scanning cameras 100 can be deployed over the samecamera hole or over multiple separate camera holes as appropriate. Inthe latter case this may take the form of one dual-scan scanning camera300 over each camera hole.

The photos and associated position and orientation data captured by oneor more scanning cameras 100 during one or more passes of a survey areamay be used to reconstruct a dense 3D surface representation of thesurvey area, textured with the captured imagery. 3D surfacereconstruction may utilize any of a number of commercially-availablephotogrammetry software packages, including Bentley Systems'ContextCapture, Agisoft's PhotoScan, and Capturing Reality'sRealityCapture.

FIG. 25 shows an activity diagram for a photogrammetry process for 3Dsurface reconstruction from data captured by one or more scanningcameras 100. An estimation step 808 accepts a stream of GNSS positions802, IMU orientations 804, and scan directions 806, and estimates theposition & orientation 810 of each captured photo 800. A reconstructionstep 812 operates on the stream of captured photos 800 and estimatedphoto positions & orientations 810, triangulating common image featuresand reconstructing a dense 3D surface 814 textured with the capturedimagery. Each scan direction 806 comprises the viewing angle 182 andviewing direction 184 of the imaging beam 160. The inputs 802, 804 and806 to the estimation step 808 are timestamped, and the estimation step708 may interpolate input values to align them in time.

The photogrammetry process intrinsically refines the estimated position& orientation 810 of each photo 800, so it is not crucial that theinitial estimates be accurate.

The photogrammetry process may also utilize other data when available,such as imagery from other cameras carried by the survey aircraft,including one or more fixed cameras capturing imagery with a differentGSD to the scanning camera, and LiDAR data captured by a LiDAR sensorcarried by the survey aircraft.

The reconstruction of 3D surfaces from aerial photos is well describedin the literature. See, for example, Furukawa and Hernandez, Multi-ViewStereo: A Tutorial, Foundations and Trends in Computer Graphics andVision, Vol. 9, No. 1-2, (2013).

A true orthomosaic may be generated from the textured 3D surface 814 byorthographically projecting the surface according to a chosen viewingdirection. The viewing direction may be nadir or oblique. Alternatively,an orthomosaic may be generated more directly by blending orthorectifiedphotos.

The creation of accurate orthomosaics from aerial photos is welldescribed in the literature. See, for example, Elements ofPhotogrammetry with Application in GIS, Fourth Edition (Wolf et al.)(McGraw-Hill 2014), and the Manual of Photogrammetry, Sixth Edition(American Society for Photogrammetry and Remote Sensing (ASPRS) 2013).

Efficient oblique imaging involves a compromise between the variation inoblique angle within the oblique imagery (corresponding to the viewingangle 182), and the variation in lateral pointing direction within theoblique imagery (corresponding to the viewing direction 184). Whenutilizing the scanning camera 100 mounted vertically, as previouslydescribed in this specification, variation in the oblique angle dependson the flightline spacing. The larger the spacing, the greater thevariation. This is a characteristic of any scanning camera used in thisway. Because the scan field 600 of the scanning camera 100 is curved, alarger spacing also leads to greater variation in the lateral pointingdirection within the oblique imagery.

If an aerial camera with a wide field of view is used to capture obliqueimagery at a fixed oblique angle, as is more conventional, then theresolution of the oblique imagery decreases laterally with increasingdistance from the center of the field of view, i.e. as the distance tothe ground increases. The curved field of view of the scanning camera100 offers a superior alternative, and the scanning camera 100 can bemounted at an oblique angle for the purpose of dedicated obliqueimaging.

FIG. 26A shows the scanning camera 100 tilted at 45 degrees for obliqueimaging. As an alternative, FIG. 26B shows the scanning camera 100tilted at 22.5 degrees and its scanning mirror 122 tilted at 22.5degrees from the spin axis 126 for 45-degree oblique imaging. As afurther alternative, FIG. 26C shows the scanning camera 100 with asingle mirror, i.e. with a scanning mirror 122 but no correction mirror112, tilted at 45 degrees for oblique imaging.

The scanning mirror 122 of an upright scanning camera 100 can also betilted at 67.5 degrees to the optical axis to effect oblique imaging.Any interference between the correction mirror 112 and the obliqueimaging beam 160 can be ameliorated by increasing the separation of thecorrection mirror 112 and the scanning mirror 122. Alternatively, thetilt of the correction mirror can be increased to bend the optical axis106 downwards, and the scanning mirror 122 can be translated downwardsaccordingly. The tilt of the scanning mirror 122 must then also beincreased to effect the desired oblique angle of the scanning opticalaxis 108.

FIG. 27 shows a wide dual oblique scan field 650 of a dual-scan obliquescanning camera comprising two scanning cameras 100 tilted at 45 degrees(as shown in FIG. 26A or FIG. 26C), one scanning camera 100 b facingforward and the other scanning camera 100 f facing backward. The dualoblique scan field 550 comprises a forward oblique scan field 540 f anda backward oblique scan field 540 b. Note that the forward oblique scanfield 540 f is captured by a backward-facing scanning camera 100 f thatis tilted forward, and the backward oblique scan field 540 b is capturedby a forward-facing scanning camera 100 b that is tilted backward.

High-quality oblique imagery captured using a wide-field oblique camerais most efficiently captured in two orthogonal passes, as shown in FIG.9B, rather than imposing an inefficiently narrow flightline spacing.

FIG. 28A shows a narrow scan field 650 of a triple-scan oblique scanningcamera 500, comprising two scanning cameras 100 f and 100 b tilted at 45degrees for oblique imaging, and one scanning camera 100 n mountedvertically for nadir imaging. FIG. 28B shows two overlapping scan fields650 a and 650 b of the triple-scan oblique scanning camera 500 from twosurvey passes with orthogonal headings 402.

FIG. 29A shows two adjacent scan fields 650 a and 650 b of a dual-scanoblique scanning camera 500 from adjacent flightlines (i.e. with nadirimaging omitted for clarity). FIG. 29B shows four scan fields 650 a, 650b, 650 c and 650 d of the dual-scan oblique scanning camera 500 fromadjacent flightlines within two orthogonal passes 404 and 406 of anaerial survey.

FIG. 30A shows the intersection of corner sensor point beams 170 withthe scanning mirror 122 of the oblique scanning camera 100 at an extremespin angle, and the resultant size of the scanning mirror 122. FIG. 30Bshows the intersection of corner sensor point beams with the scanningmirror of the oblique scanning camera 100 through a full scan range.Since the extreme spin angle for dedicated oblique imaging is smallerthan the extreme spin angle for full-field oblique and nadir imaging (asillustrated by FIGS. 8A and 8B), the required size of the scanningmirror 122 can be significantly smaller.

FIG. 31 shows a perspective view of the scan field 650 of thetriple-scan oblique scanning camera 500 carried by a survey aircraft400.

FIG. 32A shows a perspective view of the scanning camera 100 configuredwith a horizontal spin axis 126 consistent with the prior art, resultingin a linear side-to-side scan path. This scanning camera is hereafterreferred to as a linear scanning camera 900. When the spin axis 126coincides with the camera optical axis 106, the scanning mirror 122 canbe limited to a cylindrical cross section independent of spin angle 128,i.e. an ellipse as shown.

FIG. 32B shows a scan field 910 of the linear scanning camera 900operated at an altitude of 14,000 feet. The linear scanning camera 900is shown using a rectangular image sensor 144 (a CMOSIS CMV50000 with7920×6004 square 4.6 um pixels) to illustrate the effect of arectangular rather than square image sensor on the scan field 910.

FIG. 32C shows the scan field 910 with the image sensor 144 rotated 45degrees so that oblique images are captured squarely, rather thanrotated as in FIG. 32B, to better meet market expectations for obliqueimages. However, due to the rectangular shape of the image sensor andthe nature of the linear scan field 910, its sensor fields 162 have alandscape aspect at one end of the scan field and a portrait aspect atthe other end, which is not ideal.

Prior-art scanning aerial camera systems are known to capture four-waynadir and oblique imagery in a crossed (X-shaped) pattern comprising twolinear scan fields at right angles to each other (and rotated at 45degrees to the flight direction). See Lapstun et al. (U.S. Pat. No.9,641,736, “Wide-area aerial camera systems”), the contents of which areherein incorporated by cross reference.

FIG. 33A shows the dual scan field 610 of the dual-scan scanning camera100 using the larger rectangular CMV50000 image sensor 144, while FIG.33B shows a crossed scan field 912, comprising two linear scan fields910 a and 910 b, of a dual-scan linear scanning camera comprising twolinear scanning cameras 900. In contrast to the crossed scan field 912,in the dual scan field 610 all four extreme oblique images have the sameaspect.

Omitting half of the nadir portion of one scan field and half of thenadir portion of the other scan field, as discussed in relation to FIGS.19 through 21, also works to lower the average shot rate and increaseefficiency when using crossed linear scan fields.

An alternative way to achieve a curved or shaped scan path 180, with anadir viewing angle in the center and a progressively more obliqueviewing angle towards each end, is to augment a linear scanningmechanism with a variable deflection mechanism, whereby, during thescan, the imaging beam is, for example, progressively deflected in adirection perpendicular to the nominal linear scan axis as the scanprogresses away from the nadir point, thus inducing the desired shape inthe scan path. The shape of the scan path is controlled by therelationship between the scan angle of the linear scanning mechanism andthe deflection angle of the deflection mechanism, and an arbitrary shapemay be induced by suitable control of the deflection mechanism.

Several prior-art linear scanning mechanisms exist. Lapstun et al. (U.S.Pat. No. 9,641,736, “Wide-area aerial camera systems”), the contents ofwhich are herein incorporated by cross reference, describes linearscanning effected by spinning a mirror about a horizontal spin axis. Italso describes linear scanning effected by progressively tilting amirror in the scan direction. Cope et al. (U.S. patent application Ser.No. 15/513,538, “An aerial camera system”), the contents of which areherein incorporated by cross reference, describes linear scanningeffected by spinning an entire camera about a horizontal spin axis, andreflecting the imaging beam towards the ground via a mirror. Pechatnikovet al. (U.S. patent application Ser. No. 11/607,511, “Digital mappingsystem based on continuous scanning line of sight”), the contents ofwhich are herein incorporated by cross reference, describes linearscanning effected by swinging an entire camera back and forth along thescan path.

If a linear scanning mechanism is used for fixed oblique imaging, i.e.where the aim is to tilt the entire scan field at an oblique angle, itmay comprise a fixed deflection mirror for this purpose, which typicallybends the optical path by 135 degrees. For example, in Cope el al. asteering mirror (which also used for motion compensation) serves thispurpose for its two oblique cameras. This also has the effect ofinducing a curve in the resultant oblique scan field. Note that it doesnot, however, yield a scan field with a nadir viewing angle in thecenter. Rather, the minimum viewing angle within the scan field equalsthe fixed deflection angle. If, on the other hand, fixed oblique imagingis achieved by tilting the entire scanning mechanism, then no curve isinduced in the resultant oblique scan field.

In general, the curve-inducing deflection mechanism may be made to acton an assembly comprising the camera and linear scanning mechanism, oron an assembly comprising the linear scanning mechanism, or on asub-assembly on which the linear scanning mechanism itself acts. Thecurve-inducing deflection mechanism may comprise any suitable actuatoror motor that can induce the required deflection at the required rate.It may, for example, comprise a stepper motor or piezo-electric actuatoror motor. It may also, where necessary, comprise additional componentssuch as a mirror.

FIG. 34A shows a schematic of a linear scanning camera 900 with ascanning mirror 122 spinning about a horizontal spin axis 126, e.g. asdescribed in relation to FIG. 32A, and consistent with the prior art(e.g. as described in Lapstun et al.). The camera assembly 902 comprisesthe camera 102 and lens assembly 104. The linear scanning camera 900scans the imaging beam 160 along a linear scan path parallel to the yaxis. The optional correction mirror 112 is omitted for clarity, and thecamera assembly 902 is therefore horizontal.

FIG. 34B shows a steerable linear scanning camera 920 based on thespinning-mirror linear scanning camera 900 of FIG. 34A. The scanningcamera 900 is augmented with a tilting mirror drive 924 acting on thescanning mirror 122 to rotate it about the y axis, resulting in a tiltedimaging beam 926. The tilting mirror drive 924 is coupled to and actedupon by the scanning mirror drive 120, and may comprise any suitableactuator or motor. The tilted imaging beam 926 may be made to follow anarbitrary shaped scan path by varying its tilt angle during a scan.

A tilting mirror drive 924 may also be added to the scanning camera 100in the same way, either to allow the tilt of the scanning mirror 122 tobe varied during a scan to finely adjust the shape of the scan path 180,or to allow the fixed tilt of the scanning mirror 122 to be changed toadjust the overall flatness of the curve of the scan path 180.

FIG. 34C shows a linear scanning camera 900 with a camera assembly 902spinning about a horizontal axis 906, consistent with the prior art(e.g. as described in Cope et al.). It comprises a scanning camera drive904, which may comprise any suitable actuator or motor, coupled to thecamera assembly 902, and a fixed mirror 908 for deflecting the imagingbeam 160 downwards.

FIG. 34D shows a steerable scanning camera 920 based on thespinning-camera linear scanning camera 900 of FIG. 34C. The scanningcamera 900 is augmented with a tilting mirror 922 (replacing the fixedmirror 908), and a tilting mirror drive 924 acting on the tilting mirror922 to rotate it about the y axis, resulting in a tilted imaging beam926. The tilting mirror 922 and tilting mirror drive 924 are coupled tothe camera assembly 902 and hence are acted upon by the scanning cameradrive 902.

FIG. 34E shows a linear scanning camera 900 with a camera assembly 902spinning (or swinging) about a vertical axis 906, consistent with theprior art (e.g. as described in Pechatnikov et al.). It comprises ascanning camera drive 904, which may comprise any suitable actuator ormotor, coupled to the camera assembly 902, and a fixed mirror 908 fordeflecting the imaging beam 160 downwards. In Pechatnikov et al. thecamera assembly 902 faces downwards, swings about a horizontal axis, andthere is no need for a fixed mirror 908. The near-equivalent ofPechatnikov et al. in FIG. 34E is used for consistency with thesubsequent augmentation shown in FIG. 34F.

FIG. 34F shows a steerable scanning camera 920 based on theswinging-camera linear scanning camera 900 of FIG. 34E. The scanningcamera 900 is augmented with a tilting mirror 922 (replacing the fixedmirror 908), and a tilting mirror drive 924 acting on the tilting mirror922 to rotate it about the y axis, resulting in a tilted imaging beam926. The tilting mirror 922 and tilting mirror drive 924 are coupled tothe camera assembly 902 and hence are acted upon by the scanning cameradrive 902.

The tilting mirror drive 924 may be coupled to the side or the back ofthe mirror 122 or 922. Ensuring the axis of rotation is at, or close to,the face of the mirror minimizes the required size of the mirror.

FIG. 35A shows a shaped scan field 940 of a steerable scanning camera920. The illustrative shape is a “V” shape induced by varying the tiltangle of the scanning mirror 122 (or tilting mirror 922), via thetilting mirror drive 924, as a function of the spin angle 128. AV-shaped scan field has the advantage that the viewing direction 184 canbe a constant (plus or minus) 45 degrees in each arm of the scan field.

FIG. 35B shows a dual scan field 950 of a dual-scan steerable scanningcamera 930. The X-shaped dual shaped scan field 950 comprises a forwardV-shaped scan field 940 f and a backward V-shaped scan field 940 b. Thedual-scan steerable scanning camera 930 comprises a forward-facingsteerable scanning camera 920 f and a backward-facing steerable scanningcamera 920 b.

In general, any number of scanning cameras 920 may be deployed, suitablyrotated (e.g. to ensure evenly-spaced viewing directions), to capture adesired number of views. Any number of scanning cameras 920 may also bedeployed, with the same rotation, to increase capture throughput.

Another scan-field shape of interest has a constant 45-degree viewingdirection 184 within each oblique sub-field 622, but a zero-degreeviewing direction within the nadir sub-field 620.

FIG. 36 lists the equations relating the viewing angle (t) 182 and theviewing direction (p) 184 of a steerable scanning camera 920 to the spinangle (a) and the additional tilt angle (b) of the imaging beam 926(i.e. the tilt angle beyond the nominal angle of the imaging beam 160),as induced by the tilt of the mirror 122 or 922. EQ1, EQ2 and EQ3 givethe direct relations. EQ4, based on EQ1, and EQ5, based on EQ3, give thevalues for the spin angle (a) and the imaging beam tilt (b)corresponding to a specific viewing angle (t) of 45 degrees and aspecific viewing direction (p) of 45 degrees. The tilt of the mirror 122or 922 is half the tilt (b) of the imaging beam. EQ6, derived from EQ1and EQ2, gives the formula for the imaging beam tilt (b) that results ina constant viewing direction (p) of 45 degrees throughout the scan, i.e.corresponding to the V-shaped scan field shown in FIG. 35A.Alternatively, using a constant ratio of (b) to (a) of approximately0.85, based on their values from EQ4 and EQ5, yields an almost constantviewing direction (p) within a few degrees of 45 degrees throughout thescan.

FIG. 37 shows a perspective view of the dual shaped scan field 950 ofthe dual-scan steerable scanning camera 930 carried by a survey aircraft400.

A single camera assembly can be time-multiplexed between multiplescanning mechanisms, e.g. to realize a more compact mechanical design,if the resultant reduced aggregate shot rate is adequate for aparticular application.

As shown in FIGS. 38A and 38B, a dual-scan scanning camera 300 can berealized using two scanning mechanisms, each comprising a scanningmirror drive 120 and a scanning mirror 122 as previously described inrelation to the scanning camera 100, sharing a single camera assembly902. The camera assembly 902 is time-multiplexed between the twoscanning mechanisms by a multiplexing mirror 960. In FIG. 38A themultiplexing mirror rotates (or spins) about the z axis between twooperative positions, whereas in FIG. 38B it rotates (or tilts) about they axis. The multiplexing mirror 960 is coupled to a multiplexing mirrordrive 962 that rotates the mirror. The multiplexing mirror drive 962 maybe of any suitable type, e.g. as previously described in relation to thescanning mirror drive 120 or the tilting mirror drive 924. Themultiplexing mirror 960 may also perform the functions of the correctionmirror 112, and the multiplexing mirror drive may perform the functionsof the correction mirror stage 110. Alternatively, the correction mirrorstage 110 and/or the correction mirror 112 may be provided separately.

FIG. 38C similarly shows a dual-scan oblique scanning camera 300realized using two scanning mechanisms sharing a single camera assembly902. Each scanning camera 122 is shown tilted at 67.5 degrees to theoptical axis to reflect a 45-degree imaging beam 160 into the camera.Paired with another scanning camera 100, the dual-scan oblique scanningcamera 300 can be used to realize a triple-scan scanning camera 500 aspreviously described.

FIG. 38D shows a dual-scan steerable scanning camera 930 realized usingtwo linear scanning mechanisms, each comprising a scanning mirror drive120, a scanning mirror 122, and a tilting mirror drive 924, aspreviously described in relation to the steerable scanning camera 920,sharing a single camera assembly 902.

The present invention has been described with reference to a number ofpreferred embodiments. Other embodiments will be obvious to someone ofordinary skill in the art, and the scope of the invention is limitedonly by the attached claims.

The invention claimed is:
 1. An aerial camera for capturing a set ofimages along at least two curved scan paths on an object plane within anarea of interest, each image of the set of images associated with aviewing angle and a viewing direction relative to the object plane, theaerial camera comprising a scanning camera associated with each scanpath, each scanning camera having a camera optical axis and comprising:(a) an image sensor; (b) a lens; (c) a scanning mirror; and (d) a drivecoupled to the scanning mirror; wherein: (i) the drive is operative torotate the scanning mirror about a spin axis according to a spin angle;(ii) the spin axis is tilted relative to the camera optical axis; (iii)the scanning mirror is tilted relative to both the camera optical axisand the spin axis, and is positioned to reflect an imaging beam into thelens, a viewing angle and a viewing direction of the imaging beamrelative to the object plane varying with the spin angle and a pointingdirection of the camera optical axis; (iv) the lens is positioned tofocus the imaging beam onto the image sensor; and (v) the image sensoris operative to capture each image along the scan path by sampling theimaging beam at a value of the spin angle corresponding to the viewingangle and the viewing direction of the image.
 2. The aerial camera ofclaim 1, wherein each spin axis is substantially orthogonal to itsrespective camera optical axis.
 3. The aerial camera of claim 1, whereineach scanning mirror it tilted at approximately 45 degrees to itsrespective camera optical axis when its respective spin angle is zero.4. The aerial camera of claim 1, wherein the set of images comprises atleast some oblique images with substantially orthogonal viewingdirections.
 5. The aerial camera of claim 1, wherein the set of imagescomprises at least one image with a substantially nadir viewing angleand a plurality of images with substantially oblique viewing angles. 6.The aerial camera of claim 1, wherein each scanning camera comprises acorrection mirror positioned to bend its camera optical axis between itslens and its scanning mirror.
 7. The aerial camera of claim 6, whereineach correction mirror is tilted at approximately 45 degrees to itsrespective camera optical axis, thereby to bend the camera optical axisby approximately 90 degrees.
 8. The aerial camera of claim 6, furthercomprising a correction mirror stage coupled to each correction mirror,the correction mirror stage operative to rotate the correction mirrorabout at least one correction axis according to at least one correctionangle.
 9. The aerial camera according to claim 1, wherein the cameraoptical axes of at least two of the scanning cameras point in oppositedirections to each other.
 10. The aerial camera according to claim 1,wherein the camera optical axes of at least two of the scanning cameraspoint in directions that are perpendicular to each other.
 11. The aerialcamera according to claim 1, the aerial camera comprising at least threescanning cameras, wherein the camera optical axes of a first two of thescanning cameras point in opposite directions to each other, and thecamera optical axis of a third scanning camera points in a directionperpendicular to the first two pointing directions.
 12. A method ofcapturing, using the aerial camera of claim 1, the set of images alongthe at least two curved scan paths within the area of interest, themethod comprising, for each scan path, rotating, using the respectivedrive, the respective scanning mirror about its spin axis according to avalue of the spin angle corresponding to the viewing angle and theviewing direction of the image, and capturing, using the image sensorassociated with the scan path, the image.
 13. The method of claim 12,wherein the set of images comprises at least some oblique images withsubstantially orthogonal viewing directions.
 14. The method of claim 12,wherein each spin axis is substantially orthogonal to its respectivecamera optical axis.
 15. The method of claim 12, wherein each scanningmirror it tilted at approximately 45 degrees to its respective cameraoptical axis when the spin angle is zero.
 16. The method of claim 12,the method further comprising moving the aerial camera along a surveypath above the area of interest, and capturing, within each of a set ofselected intervals along the survey path and using the aerial camera, acorresponding one of a plurality of the sets of images.
 17. The methodof claim 16, wherein the camera optical axes of at least two of thescanning cameras point in opposite directions to each other, and the twopointing directions are parallel to a direction of movement.
 18. Themethod of claim 16, wherein the camera optical axes of at least two ofthe scanning cameras point in opposite directions to each other, and thetwo pointing directions are perpendicular to a direction of movement.19. The method of claim 16, the aerial camera comprising at least threescanning cameras, wherein the camera optical axes of a first two of thescanning cameras point in opposite directions to each other, the cameraoptical axis of a third of the scanning cameras points in a directionperpendicular to the first two pointing directions, and the first twopointing directions are perpendicular to a direction of movement. 20.The method of claim 16, the method comprising using the first twoscanning cameras to capture images with substantially oblique viewingangles, and using the third scanning camera to capture images withsubstantially nadir viewing angles.